The present invention relates to a high-efficiency coding method, a high-efficiency coding apparatus, a recording and reproducing apparatus, and an information transmission system for use with a VCR, an information transmission system or the like.
A method using a motion compensation prediction and a DCT (discrete cosine transform) has hitherto been known as a high-efficiency coding method for an interlace component digital moving picture signal (e.g., CCIR recommendation 601, etc.).
The following three methods are known as the motion compensation prediction method.
A frame type motion compensation method:
This is a method of motion-compensating a processed image as a frame. Graphics image data generated on a computer, for example, is a non-interlace image signal. Therefore, when the graphics image data is displayed on a television monitor for displaying an NTSC television signal, such graphics image data is converted into an NTSC interlace image signal. Since the non-interlace graphics image data obtained from the computer is converted into the NTSC interlace image signal, the interlace image signal is an image signal of frame unit, and an odd-numbered scanning line and an even-numbered scanning line do not have a time difference of one field therebetween unlike an ordinary NTSC television image signal obtained from a television camera or the like. Thus, in this case, it is optimum to effect the frame type motion compensation on the processed image signal.
Field type motion compensation method:
This is a method of motion-compensating a processed image at every field. An NTSC television image signal obtained from a television camera or the like, for example, have a time difference of one field between the odd-numbered scanning line and the even-numbered scanning line of one frame. Accordingly, in this case, it is optimum to effect the field type motion compensation on the processed image signal.
A frame type/field type adaptive motion compensation method:
This is a method of adaptively switching the frame type motion compensation and the field type motion compensation.
Important points of the frame type motion compensation method and the field type motion compensation method will be described below. A "half-pel" in the frame type motion compensation and a "half-pel" in the field type motion compensation become different from each other. Specifically, in the frame type motion compensation, an average value of pixels (pels) adjacent in the longitudinal, lateral or oblique direction within frame is referred to as a "half-pel", and in the field type motion compensation, an average value of pixels adjacent in the longitudinal, lateral or oblique direction within odd field and an average value adjacent in the longitudinal, lateral or oblique direction within even field are referred to as "half-pels". Accordingly, pixel data is newly generated by averaging two pixel data adjacent in the horizontal, vertical or oblique direction in a memory, for example. If half-pel data is newly generated and a motion detection is carried out in the state including the newly-generated half-pel data, then it is possible to obtain motion vector data with accuracy higher than that of motion vector data that has been detected when at least half-pel data is not generated. Thus, accurate macroblock data can be extracted from a preceding frame, and hence a difference between macroblock data of the preceding frame and macroblock data of the present frame can be made accurate.
The number of motion vectors within one frame in the field type motion compensation becomes twice of those within one frame in the frame type motion compensation. Since it is customary that data is processed in the macroblock unit in the motion compensation, when the macroblock in the frame type motion compensation is of 16 lines.times.16 pixels, macroblocks in the odd field and the even field in the field type motion compensation each become 8 lines.times.16 pixels.
In other words, the number of macroblocks within one frame in the field type motion compensation becomes twice the number of macroblocks within one frame in the frame type motion compensation. The reason for this is that the frame type motion compensation has one motion vector with respect to one macroblock and that the field type motion compensation has one motion vector with respect to a macroblock of odd field and a macroblock of even field. Accordingly, the frame type motion compensation has one motion vector with respect to one macroblock and the field type motion compensation has two motion vectors with respect to one macroblock in the frame type motion compensation.
The frame type motion compensation and the field type motion compensation are different from each other as described above. A coding apparatus for effecting the frame/field adaptive motion compensation will be described with reference to FIG. 1 and FIGS. 2A to 2C.
FIG. 1 of the accompanying drawings shows an example of an encoding apparatus for effecting the frame/field adaptive motion compensation.
As shown in FIG. 1, this encoding apparatus comprises a blocking circuit 201 for blocking interlace image data supplied thereto from an input terminal 200 at the macroblock unit of 16 lines.times.16 pixels, an adding circuit 202 for subtracting macroblock data MB(F-1) of motion-compensated preceding frame supplied thereto from a motion compensating circuit 214 from macroblock data MB(F) of the present frame supplied thereto from the blocking circuit 201, a switch 204 for switching the macroblock data MB(F) of the present frame supplied thereto from the blocking circuit 201 and difference data supplied thereto from the adding circuit 202 based on a switching control signal supplied thereto from an inter/intra judgement circuit 203, a DCT (discrete cosine transform) circuit 205 for transforming the macroblock data MB(F) of the present frame from the switch 204 or the difference data in the form of coefficient data of DC component to high-order AC component at the block unit of 8 lines.times.8 pixels, a quantizer circuit 206 for quantizing coefficient data supplied thereto from the DCT circuit 205 at a predetermined quantization step size, a variable word-length encoder 207 for variable-word-length-encoding the coefficient data supplied thereto from the DCT circuit 205 by use of a suitable method such as a run-length-coding method or a Huffman coding method, an inverse quantizer circuit 209 for inverse-quantizing the coefficient data supplied thereto from the quantizer circuit 206 to provide the coefficient data of the DCT circuit 205, an IDCT (inverse discrete cosine transform) circuit 210 for transforming coefficient data supplied thereto from the inverse quantizer circuit 209 into the original macroblock data MB(F) of the present frame or the difference data, an adding circuit 211 for adding the output of the IDCT circuit 210 and the macroblock data MB(F-1) of the motion-compensated preceding frame supplied thereto from the motion compensating circuit 214, a switch 212 for selectively supplying the output of the adding circuit 211 and the output of the IDCT circuit 210 based on the switching control signal supplied thereto from the inter/intra judgement circuit 203, a frame memory 213 for storing an output of the switch 212 in its storage area, a frame type motion vector detecting circuit 216 for obtaining frame motion vector data mv(Fr) by motion-detecting the macroblock data MB(F-1) of the preceding frame supplied thereto from the frame memory 213 and the macroblock data MB(F) of the present frame supplied thereto from the blocking circuit 201 at the frame unit, a field type motion detecting circuit 215 for obtaining odd field motion vector data mv(odd) and even-numbered motion vector data mv(even) by motion-detecting the macroblock data MB(F) of the present frame supplied thereto from the blocking circuit 201 and the macroblock data MB(F-1) of the preceding frame supplied thereto from the frame memory 213 at the field unit, a field/frame type switching circuit 217 for selecting the odd field motion vector data mv(odd) supplied thereto from the field type motion detecting circuit 215, the even field motion vector data mv(even) supplied thereto from the field type motion detecting circuit 215 or the frame motion vector data mv(Fr) supplied thereto from the frame type motion detecting circuit 216 and a motion compensating circuit 214 for generating address data generated based on the frame motion vector data or the field motion vector data mv(Fr)/mv(Fi) supplied thereto from the field/frame type switching circuit 217, reading out the macroblock data MB(F-1) of the preceding frame from the frame memory 213, and supplying the macroblock data MB(F-1) thus read to the adding circuit 202.
The switches 204, 212 are operated in unison with each other in response to the switching control signals supplied thereto from the inter/intra judgement circuit 203. Specifically, when the macroblock data MB(F) of the present frame from the blocking circuit 201 is selected, the switch 204 connects a movable contact c to one fixed contact a, and concurrently therewith, the switch 212 connects a movable contact c to one fixed contact a. When the difference data from the adding circuit 202 is selected, the switch 204 connects the movable contact c to the other fixed contact b, and concurrently therewith, the switch 212 connects the movable contact c to the other fixed contact b.
A manner in which the macroblock data MB is generated will be described with reference to FIGS. 2A to 2C. In FIGS. 2A to 2C, "an open square" represents a pixel of odd field, and an open circle" represents a pixel of even field, respectively. Arrows in FIGS. 2A to 2C show scanning trajectories obtained in the raster scanning or block scanning.
If image data of one frame obtained in the raster scanning shown in FIG. 2A is processed by the block scanning as shown in FIG. 2C, then there is extracted macroblock data MB of 16 lines.times.16 pixels, for example, shown by broken lines in FIG. 2B. Then, the motion compensation processing is carried out by use of the resultant macroblock data MB.
The inter/intra judgement circuit 203 compares dispersed values of the macroblock data MB(F) of the present frame and the difference data, and selects one of the macroblock data MB(F) of the present frame and the difference data having a smaller dispersed value. The dispersed values are calculated by the following equations (1) and (2): ##EQU1## where MP is an average luminance level of pixel provided within the macroblock, and Pj is a luminance level of pixel. ##EQU2## where Var.sub.MBLK is a dispersed value of macroblock.
The field type motion detecting circuit 215 and the frame type motion detecting circuit 216 carry out motion-detection by block-matching which will be described later on.
The frame type motion detection and the field type motion detection will be described with reference to FIGS. 3 and 4. In FIGS. 3 and 4, "open square" represents a pixel of odd field, "open circle" represents a pixel of even field, and "open triangle" represents a half-pel, respectively.
FIG. 3 is a schematic diagram used to explain the frame type motion detection. The vertical direction in FIG. 3 represents a motion vector between a pixel of odd field and a pixel (upper and lower direction) of even field adjacent to the above pixel of odd field, and a value of this motion vector is assumed to be "1", for example (motion amount of one pixel). A motion from upper to lower direction between the preceding frame and the present frame is represented by "minus (-)", and a motion from lower to upper direction between the preceding frame and the present frame is represented by "plus (+)". If one half-pel of the one-frame preceding frame is a pixel of even field of the present frame and the other half-pel of the one-frame preceding frame is a pixel of odd field of the present frame as shown by solid arrows in FIG. 3, then values of these frame motion vectors both become "-2.5" (motion amount of 2.5 pixels from upper to lower direction).
FIG. 4 is a schematic diagram used to explain the field type motion detection. In FIG. 4, the horizontal direction represents a motion vector obtained between the pixel of odd field and the pixel (upper and lower direction) of even field adjacent to the above pixel. A value of this motion vector is assumed to be "1", for example (motion amount of one pixel). A motion from upper to lower direction between the preceding frame and the present frame is represented by "minus (-)", and a motion from lower to upper direction between the preceding frame and the present frame is represented by "plus (+)". If a half-pel of the even field of the preceding frame is a pixel of even field with respect to the even field as shown by a solid-line arrow in FIG. 4, then a value of motion vector of even field becomes "-3" (motion amount of three pixels from upper to lower direction). The above half-pel, i.e., half-pel of even field becomes a pixel of odd field in the odd field. If a pixel of odd field of one-field preceding frame is a pixel of odd field of the present frame with respect to the odd field, then a value of motion vector of odd field becomes "-4" (motion amount of four pixels from upper to lower direction).
As is clear from the above description, when coordinates of vector in the longitudinal direction are examined from a frame standpoint, the half-pel in the frame type motion compensation corresponds to "0.5", and the half-pel in the field type motion compensation corresponds to "1.0". Accordingly, the field type motion compensation has two vectors, i.e., even field vector and odd field vector with respect to one macroblock, and the frame type motion compensation has one motion vector with respect to one macroblock.
The field/frame type switching circuit 217 is supplied with the frame motion vector data mv(Fr), the odd field motion vector data mv(odd), the even field motion vector data mv(even) and difference absolute value sum data obtained when these vector data are obtained. The field/frame type switching circuit 217 compares the above three difference absolute value sum data, and selects motion vector data obtained from the difference absolute value sum data of smallest value.
The interlace image data of the present frame is supplied through the input terminal 200 to the blocking circuit 201. This interlace image data is divided by the blocking circuit 201 into macroblocks of 16 pixels.times.16 pixels, for example, and supplied to the switch 204, the adding circuit 202, the field type motion detecting circuit 215, and the field/frame type switching circuit 217, respectively.
The adding circuit 202 subtracts the motion-compensated macroblock data MB(F-1) of the preceding frame supplied thereto from the motion compensating circuit 214 from the macroblock data MB(F) of the present frame to provide difference data, and this difference data is supplied to the switch 204. The macroblock data MB(F) of the present data supplied to the switch 204 and the difference data from the adding circuit 202 are selectively supplied to the DCT circuit 205 in response to the switching control signal from the inter/intra judgement circuit 203.
The macroblock data MB(F) of the present frame or the difference data supplied to the DCT circuit 205 is transformed into coefficient data of from DC component to high-order AC component, and supplied to the quantizer circuit 206, in which it is quantized with a predetermined quantization step size. The quantized coefficient data is supplied to the variable word-length encoding circuit 207 and the inverse quantizer circuit 209. The quantized coefficient data supplied to the variable word-length encoding circuit 207 is variable-word-length-encoded by a proper method such as run-length-coding method or Huffman coding method, and outputted through the output terminal 208.
On the other hand, the quantized coefficient data supplied to the inverse quantizer circuit 209 is transformed into the original coefficient data, which is not yet quantized, and supplied to the IDCT circuit 210. The coefficient data supplied to the IDCT circuit 210 is decoded by the IDCT circuit 210 into the original macroblock data MB(F) or difference data, and supplied to the adding circuit 211 and the switch 212.
The difference data supplied to the adding circuit 211 is added to the motion-compensated macroblock data MB(F-1) of the preceding frame supplied thereto from the motion compensating circuit 214, and supplied to the switch 212. The added output and the macroblock data MB(F) of the present frame supplied to the switch 212 are selectively supplied to the frame memory 213 in response to the switching control signal from the inter/intra judgement circuit 203, and stored in a memory space of the frame memory 213.
The macroblock data MB(F-1) of the preceding frame read out from the frame memory 213 is supplied to the field type motion detecting circuit 215 and the frame type motion detecting circuit 216. The field type motion detecting circuit 215 is supplied with the macroblock data MB(F-1) of the preceding frame read out from the frame memory 213 and the macroblock data MB(F) of the present frame from the blocking circuit 201. Then, the field type motion detecting circuit 215 effects a motion detection of the field unit on the basis of the macroblock data MB(F) of the present frame and the macroblock data MB(F-1) of the preceding frame. Resultant odd-numbered motion vector data mv(odd) and even-numbered motion vector data mv(even) are supplied to the field/frame type switching circuit 217 together with the difference absolute value sum data.
The frame type motion detecting circuit 215 is supplied with the macroblock data MB(F-1) of the preceding frame read out from the frame memory 213 and the macroblock data MB(F) of the present frame from the blocking circuit 201. The frame type motion detecting circuit 216 effects a motion detection of the frame unit by the macroblock data MB(F) of the present frame and the macroblock data MB(F-1) of the preceding frame. Resultant frame motion vector data mv(Fr) is supplied to the field/frame type switching circuit 217 together with the difference absolute value sum data.
The field/frame type switching circuit 217 compares the difference absolute value sum data supplied thereto from the field type motion detecting circuit 215 and the difference absolute value sum data from the frame type motion detecting circuit 216, and selects motion vector data with a smaller value, i.e., odd field motion vector data mv(odd), even field motion vector data mv(even) or the frame motion vector data mv(Fr). The motion vector data thus selected is supplied to the motion compensating circuit 214 as the frame motion vector data mv(Fr) or the field motion vector data mv(Fi). The motion compensating circuit 214 carries out a motion compensation. Specifically, the motion compensating circuit 214 reads out the macroblock data MB(F-1) of the preceding frame from the frame memory 213 in response to address data (or control signal for controlling a memory controller (not shown)) generated on the basis of the frame motion vector data mv(Fr) or the field motion vector data mv(Fi) from the field/frame type switching circuit 217. This macroblock data MB(F-1) of the preceding frame is data indicative of the original position of the macroblock data MB(F) of the present frame supplied from the blocking circuit 201.
The motion-compensated macroblock data MB(F-1) of the preceding frame from the motion compensating circuit 214 is supplied to the adding circuit 202. Thus, the adding circuit 202 subtracts the macroblock data MB(F-1) of the preceding frame from the macroblock data MB(F) of the present frame supplied thereto from the blocking circuit 201.
An example of the field type motion detecting circuit 215 and an example of the frame type motion detecting circuit 216 shown in FIG. 1 will be described with reference to FIG. 5.
As shown in FIG. 5, the field type motion detecting circuit 215 shown in FIG. 1 comprises an odd field first motion detecting circuit 243 for detecting a motion between odd fields with an accuracy of one pixel, an odd-field second motion detecting circuit 245 for detecting a motion between odd fields with an accuracy of half-pel, an even field first motion detecting circuit 244 for detecting a motion between even fields with an accuracy of one pixel, and an even field second motion detecting circuit 247 for detecting a motion between even fields with an accuracy of half-pel.
As shown in FIG. 5, the frame type motion detecting circuit 216 shown in FIG. 1 comprises a frame first motion detecting circuit 249 for detecting a motion between frames with an accuracy of one pixel, and a frame second motion detecting circuit 250 for detecting a motion between frames with an accuracy of half-pel.
An input terminal 240 to which the macroblock data MB(F) of the present frame is supplied from the blocking circuit 201 shown in FIG. 1 is connected to input terminals of the odd field first and second motion detecting circuits 243, 245, the even field first and second motion detecting circuits 244, 247, and the frame first and second motion detecting circuits 249, 250. An input terminal 242 to which the macroblock data MB(F-1) of the preceding frame from the frame memory 213 shown in FIG. 1 is connected to input terminals of the odd field first and second motion detecting circuits 243, 245, the even field first and second motion detecting circuits 244, 247 and the frame first and second motion detecting circuits 249, 250.
The odd field first and second motion detecting circuits 243, 245 are supplied with odd field macroblock data MB(odd, F) of present frame through the input terminal 240 and odd field macroblock data MB(odd, F-1) of preceding frame through the input terminal 242. The even field first and second motion detecting circuits 244, 247 are supplied with even field macroblock data MB(even, F) of present frame through the input terminal 240 and even field macroblock data MB(even, F-1) of preceding frame through the input terminal 242. Further, the frame first and second motion detecting circuits 249, 250 are supplied with the macroblock data MB(F) of present frame through the input terminal 240 and the macroblock data MB(F-1) of preceding frame through the input terminal 242.
The odd field first motion detecting circuit 243 detects a motion vector with an accuracy of integer pixels on the basis of the odd field macroblock data MB(odd, F) of the present frame and the odd field macroblock data MB(odd, F-1) of the preceding frame, and supplies resultant odd field motion vector data MV(odd) to the odd field second motion detecting circuit 245. The odd field second motion detecting circuit 245 detects a motion vector with an accuracy of half-pel on the basis of the odd field motion vector data MV(odd), the odd field macroblock data MB(odd, F) of present frame and the odd field macroblock data MB(odd, F-1) of preceding frame supplied thereto from the odd field first motion detecting circuit 243, and supplies resultant odd field motion vector data mv(odd) through the output terminal 246 to the field/frame type switching circuit 217 shown in FIG. 1.
The even field first motion detecting circuit 244 detects a motion vector with an accuracy of integer pixels on the basis of the even field macroblock data MB(even, F) of present frame and the even field macroblock data MB(even, F-1) of preceding frame, and supplies resultant even field motion vector data MV(even) to the even field type motion detecting circuit 247. The even field second motion detecting circuit 247 detects a motion with an accuracy of half-pel on the basis of the even field motion vector data MB(even), the even field macroblock data MB(even, F) of the present frame and the even field macroblock data MV(even, F-1) of the preceding frame supplied thereto from the even field first motion detecting circuit 244, and supplies resultant even field motion vector data mv(even) through the output terminal 248 to the field/frame type switching circuit 217 shown in FIG. 1.
The field type motion detection will be described more fully with reference to FIG. 6. In FIG. 6, only the eeven field type motion detection will be described, and the even field type motion detection is not shown and need not be described because only an object to be detected is different. The motion vector detection will be described later on with reference to FIG. 9, and concepts in detecting motion with accuracy of integer pixels and accuracy of half-pel will be described below.
In FIG. 6, the vertical axis represents coordinates of motion vector in the vertical direction, and horizontal axis represents coordinates of motion vector in the horizontal direction. In FIG. 6, "open circle" represents pixel data of even field, and "open triangle" represents half-pel data, respectively. FIG. 6 shows not only pixel data of the even and odd fields but also pixel data of even and odd fields of present frame and pixel data of even and odd fields of preceding frame in an overlapping state.
In this case, let us describe the field type motion detection on the assumption that the position of even field pixel data Pc of preceding frame coincides with the position of even field pixel data P(F) of present frame after one frame. If the even field pixel data P(F) of present frame is set to be a starting point, then coordinates of motion vector data MV1 with accuracy of integer pixels provided between the even field pixel data P(F) of present frame and even field pixel data Pc of preceding frame become (-1, -2) as shown by arrows and numerical values in FIG. 6, wherein "-1" is in the horizontal direction, and "-2" is in the vertical direction. In other words, the even field first motion detecting circuit 244 shown in FIG. 5 detects a motion of even field pixel data, and an accuracy of a resultant motion vector data MV1 becomes an accuracy of integer pixels.
The even field second motion detecting circuit 247 shown in FIG. 5 sets a search area (shown by a broken-line area in FIG. 6) about the even field pixel data Pc on the basis of the motion vector data MV1 {even field motion vector data MV(even) in FIG. 5} with an accuracy of integer pixels calculated by the even field first motion detecting circuit 244. Within that area, the even field second motion detecting circuit 247 detects a motion including half-pel generated from the even field pixel data, and detects motion vector data mv2 {even field motion vector data mv(even) in FIG. 5}.
If the position of even field half-pel data Pa of preceding frame is agreed with the position of even field pixel data P(F) of present frame, coordinates of motion vector data MV1 with accuracy of integer pixels become (-1, -2). In this case, a motion detection with an accuracy of half-pel is carried out, and consequently, coordinates of the motion vector data mv2 becomes (-0.5, -2). Specifically, in the first step, a motion detection with an accuracy of integer pixels is carried out, and the search area for effecting motion detection with an accuracy of half-pel is set on the basis of the resultant motion vector data in the next step. Then, a motion detection with an accuracy of half-pel is carried out, and a motion detection with a higher accuracy can be carried out.
On the other hand, the frame first motion detecting circuit 249 detects a motion with an accuracy of integer pixels on the basis of the macroblock data MB(F) of present frame and the macroblock data MB(F-1) of preceding frame, and supplies resultant frame motion vector data MV(Fr) to the frame second motion detecting circuit 250. The frame second motion detecting circuit 250 detects a motion with an accuracy of half-pel on the basis of the macroblock data MB(F) of present frame, the macroblock data MB(F-1) of preceding frame, and the frame motion vector data MV(Fr) supplied thereto from the frame first motion detecting circuit 249, and obtains resultant frame motion vector data with an accuracy of half-pel. The frame second motion detecting circuit 250 supplies this motion vector data mv (Fr) through the output terminal 251 to the field/frame type motion detecting circuit 217 shown in FIG. 1.
The frame type motion detection will be described with reference to FIG. 7. A manner in which a motion vector itself is detected will be described later on with reference to FIG. 9, and only concepts in detecting motion with an accuracy of integer pixels and with an accuracy of half-pel will be described below
In FIG. 7, the vertical axis represents coordinates of motion vector in the vertical direction, and the horizontal axis represents coordinates of motion vector in the horizontal direction, respectively. In FIG. 7, "open square" represents odd field pixel data, "open circle" represents even field pixel data, and "open triangle" represents half-pel data, respectively. FIG. 7 shows not only pixel data of the present frame but also pixel data of present frame and pixel data of preceding frame in an overlapping state.
In the following description, let it be assumed that the position of pixel data Pc of preceding frame is agreed with the position of pixel data P(F) of present frame after one frame. If the pixel data P(F) of present frame is set to a starting point, then coordinates of motion vector data MV1 with an accuracy of integer pixels between the pixel data P(F) of present frame and the pixel data Pc of preceding frame are set to (-1, -2) as shown by arrows and numerical values in FIG. 7, wherein "-1" is set on the horizontal direction, and "-2" is set on the vertical direction. Specifically, the frame first motion detecting circuit 249 shown in FIG. 5 detects a motion on the basis of whole data provided within the frame, and an accuracy of resultant motion vector data MV1 becomes an accuracy of integer pixels.
The frame second motion detecting circuit 250 sets a search area (area encircled by a broken line in FIG. 7) about the pixel data Pc on the basis of motion vector data MV1 {frame motion vector data MV(Fr) in FIG. 5)} with an accuracy of integer pixels calculated by the frame first motion detecting circuit 249. Then, within the above area, the frame second motion detecting circuit 250 detects a motion with an accuracy including half-pel generated from pixel data, and detects motion vector data mv2 {frame motion vector data mv(Fr) in FIG. 5}.
If the position of half-pel data Pa of preceding frame is agreed with the position of even field pixel data P(F) of present frame after one frame, then coordinates of the motion vector data MV1 with an accuracy of integer pixels become (-1, -2). In this case, a motion is detected with an accuracy of half-pel, and hence resultant motion vector data mv2 is set to (-1.5, -1.5). In other words, in the first step, a motion detection with an accuracy of integer pixels is carried out, and the search area for detecting a motion with an accuracy of half-pel in the next step is set on the basis of the resultant motion vector data. Then, a motion detection with an accuracy of half-pel is carried out within the above search area, thereby detecting a motion with a higher accuracy.
The frame type motion detection will be further described with reference to FIG. 8. FIG. 8 illustrates a macroblock of 16 lines.times.16 pixels, by way of example. In FIG. 8, similarly to FIG. 7, "open square" represents odd field pixel data, "open circle" represents even field pixel data, "open triangle" represents interpolated half-pel data, and vm1 to vm9 represent motion vector data with an accuracy of half-pel corresponding to positions of respective half-pels, respectively. In FIG. 8, macroblock data MB(VM5) is predicted on the basis of frame motion vector data VM5 {corresponding to the frame motion vector data MV(Fr) detected by the first motion detecting circuit 249 shown in FIG. 5}. Accordingly, the pixel data Pc shown in FIG. 7 corresponds to pixel data Pc shown in FIG. 8.
When the frame motion vector data MV5 detected by the frame first motion detecting circuit 249 is supplied to the frame second motion detecting circuit 250, the frame second motion detecting circuit 250 generates half-pel data provided around the pixel data Pc on the basis of the frame motion vector data MV5 with an accuracy of integer pixels, and detects a motion within that search area. As a result, macroblock data MB(vm3) is predicted as shown in FIG. 8. The macroblock data MB(vm3) is predicted by the frame motion vector data vm3 with an accuracy of half-pel. The half-pel data Pa shown in FIG. 8 corresponds to the half-pel data Pa shown in FIG. 7.
An example of motion detecting circuit shown in FIG. 5 will be described with reference to FIG. 9. A motion detecting circuit shown in FIG. 9 detects a motion by use of block-matching method. A frame memory provided within the blocking circuit 201 shown in FIG. 5 corresponds to a present frame memory 321 shown in FIG. 9, and the frame memory 213 shown in FIG. 1 corresponds to a reference frame memory 323 shown in FIG. 9. While the field first motion detecting circuits 242, 244, the field second motion detecting circuits 245, 247, the frame first motion detecting circuit 249, and the frame second motion detecting circuit 250 are illustrated in FIG. 5, in the case of the field type motion detecting circuit, reference blocks and remarkable blocks, which will be described below, are set at every field. In the case of the frame motion detecting circuit, reference blocks and remarkable blocks, which will be described below, are set every frame.
The motion detecting circuit shown in FIG. 9 comprises a present frame memory 321 for storing therein image data of present frame, a reference frame memory 323 for storing therein image data of preceding frame (reference frame), an address shift circuit 333 for sequentially supplying different address data to the reference frame memory 323, an adding circuit 324 for subtracting macroblock data MB(F-1) supplied thereto from the reference frame memory 323 from macroblock data MB(F) of present frame supplied thereto from the present frame memory 321, an absolute value generating circuit 325 for generating an absolute value of a subtracted result supplied thereto from the adding circuit 324, a latch circuit 327 for latching absolute value data supplied thereto from the absolute value generating circuit 325, an adding circuit 326 for adding an output of the absolute value generating circuit 325 and a latched output of the latch circuit 327 to provide difference absolute value sum data, a memory 328 for storing therein the difference absolute value sum data supplied thereto from the adding circuit 326, a minimum value detecting circuit 329 for detecting a minimum value from the difference absolute value sum data stored in the memory 328, a motion vector detecting circuit 330 for obtaining vector data based on the minimum difference absolute value sum data supplied thereto from the minimum value detecting circuit 329 and supplying the vector data to a controller 332 and the field/frame switching circuit 217 shown in FIG. 1, and the controller 332 for controlling the address shift circuit 333 based on the motion vector data supplied thereto from the motion vector detecting circuit 330 and controlling writing of image data in the present frame memory 321 and reading of stored image data from the present frame memory 321.
The motion vector detecting circuit 330 converts inputted difference absolute value sum data into motion vector data by reading out motion vector data corresponding to inputted difference absolute value sum data, e.g., data indicative of moving amount in the longitudinal and lateral directions from a suitable memory means, such as a ROM (read-only memory).
Pixel data of macroblock (8.times.8 pixels or 6.times.16 pixels) is sequentially read out from the present frame memory 321 as remarkable block under control of the controller 332. On the other hand, the address shift circuit 333 sets a search area on the memory space of the reference frame memory 323 under control of the controller 332, and sets a reference block of the same size as that of the above macroblock within the search area. Then, the address shift circuit 333 sequentially supplies address data, which is used to sequentially read out pixel data from the reference block, to the reference frame memory 323. After the reading of pixel data from the set reference block is completed, the address shift circuit 333 shift the position of the reference block by one pixel within the search area by supplying address data to the reference frame memory 323, and reads out pixel data within the reference block shifted by one pixel by sequentially supplying address data to the reference frame memory 323.
The adding circuit 324 subtracts pixel data provided within the reference block read out from the reference frame memory 323 from pixel data provided within the remarkable block read out from the present frame memory 321. A subtracted result from the adding circuit 324 is supplied to the absolute value generating circuit 325, in which it is converted to absolute value data, and supplied through the adding circuit 326 to the latch circuit 327. The latch circuit 327 latches an added result supplied thereto from the adding circuit 326, i.e., difference absolute value sum data, whereby difference absolute value sum data between the remarkable block provided within the present frame memory 321 and one reference block provided within the reference frame memory 323 is sequentially stored in the memory 328. Then, the memory 328 finally stores therein difference absolute value sum data of the number corresponding to a number of remarkable blocks that have been sequentially set in the search area with a displacement of one pixel.
After calculation of pixel data of reference block provided within one search area and pixel data of remarkable block is finished completely, the minimum value detecting circuit 329 selects the minimum difference absolute value sum data from all difference absolute value sum data stored in the memory 328, and supplies the selected minimum absolute value sum data to the motion vector detecting circuit 330. Also, the minimum value detecting circuit 329 supplies a control signal to the controller 332 such that the controller 332 starts the next processing.
The difference absolute value sum data from the minimum value detecting circuit 329 is supplied to the motion vector detecting circuit 330. The motion vector detecting circuit 330 detects motion vector data corresponding to the difference absolute value sum data supplied thereto from the minimum value detecting circuit 329. The motion vector data obtained by the motion vector detecting circuit 330 is supplied through an output terminal 331 to the field/frame switching circuit 217 shown in FIG. 1 and the controller 332. Then, the controller 332 sets a search area in a similar manner, and controls the address shift circuit 333 and the present frame memory 321 in such a manner that pixel data provided within the reference block and pixel data provided within the remarkable block are processed one more time.
FIG. 10 shows a concept of manner in which a motion is detected according to the block-matching method. In FIG. 10, MB1 represents a macroblock of present frame, SA represents a search area of preceding frame, SIT represents a position of the macroblock MB1 in the preceding frame, and MB2 represents a macroblock of preceding frame composed of the same data as that of the macroblock MB1 of present frame.
Let it be assumed that the position of the macroblock MB1 of present frame is agreed with the position SIT shown by a broken line in the preceding frame and that data of the macroblock MB1 of present frame is data of the macroblock MB2 in the preceding frame.
In such case, a difference absolute value between data of the macroblock obtained from the search area SA by shifting address one pixel and data of the macroblock MB1 of present frame is sequentially calculated, and this difference absolute value data is added to each block, whereafter the motion vector MV is calculated based on the minimum difference absolute value sum data within difference absolute value sum data of each block. Thus, as shown in FIG. 10, there can be obtained a position of the macroblock MB1 of present frame in the preceding frame, i.e., a position of the macroblock MB2 in the preceding frame. In this example, coordinates are (-2, 4), wherein "-2" represents a moving amount of horizontal direction within the preceding frame, and "4" represents a moving amount of vertical direction in the preceding frame. Further, "minus (-1)" represents a movement from upper to lower direction in the vertical direction, and "plus (+)" represents a movement from lower to upper direction in the vertical direction.
U.S. Pat. No. 4,937,666 describes a technique concerning 2-step vector search in which a motion detection with an accuracy of half-pel is carried out after a motion detection with an accuracy of integer pixels has been carried out. U.S. Pat. No. 4,897,720 describes a technique concerning a block-matching.
When a video camera picks up an object and generates a video signal, an electrical signal which results from receiving light from the object by a CCD (charge-coupled device) is converted into a video signal of even field and an electrical signal which results from receiving light from the object by the CCD next is converted into a video signal of odd field. Thus, the video signal of odd field and the video signal of even field constitute a video signal of interlaced system. That is, the video signal of odd field and the video signal of even field have a time difference of one field therebetween. If the object is moved during a time period of one field, a motion occurs between the video signal of odd field and the video signal of even field, i.e., there occurs a difference of luminance level between corresponding pixels.
Accordingly, when a video signal obtained from a device which outputs a video signal having a time difference between the odd field and the even field is frame-type-motion-detected by half-pel and frame-type-motion-compensated, there arise the following problems. That is, when two pixel data adjacent in the vertical direction are averaged in order to generate half-pel data, pixel data of odd field and pixel data of even field are averaged and resultant average value is used as half-pel data. Therefore, when the odd field and the even field have a motion therebetween, the half-pel data includes motion of odd field and even field, i.e., a fluctuation amount of level. Accordingly, an error occurs in the motion detection of half-pel accuracy using half-pel data and a reliable motion compensation cannot be made. As a result, an error occurs in encoding for transmitting or recording a video signal, and hence information cannot be transmitted or recorded accurately.
On the other hand, when the frame type motion compensation is carried out after the field type motion detection was carried out, motion vector data for odd field and even field are respectively obtained, thereby increasing an encoding amount.
Further, in the case of frame type/field-type adaptive motion compensation wherein motion vector data obtained by the field type motion detection and the frame type motion detection are selected and a motion compensation is carried out on the basis of the selected motion vector data, as is clear from the description of FIG. 5, there have to be prepared two detecting circuits of large circuit scale for the field type motion detection and the frame type motion detection, resulting in the circuit scale being increased.
In order to carry out motion detection with half-pel accuracy, there have to be generated nine half-pel data around integer pixel data including center half-pel data. When nine half-pel data around the integer pixel data Pc shown in FIG. 5 are generated together with the center half-pel data, in addition to respective integer pixel data of the macroblock data MB(VM5), integer pixel data for generating the above-mentioned nine pixel data have to be read out from the frame memory 113. When the macroblock is of 16 lines.times.16 pixels, for example, pixel data of at least 18 lines.times.18 pixels have to be read out from the frame memory 113. Accordingly, the pixel data for generating the half-pel data have to be read out from the frame memory 113 during a limited time period. There is then the problem that a processing speed in motion detection is lowered.